Keyframe-Based User Interfaces for Digital Video

A s video use expands from entertainment to other domains—such as business, education , and personal applications—these new uses demand new forms of access. For entertainment, the typical user will happily view video linearly, making current access methods— such as fast forward and rewind—sufficient. However, in other applications, users may need to skip around to seek specific video segments. For example, a user may want to find a particular meeting in a database containing a number of recorded meetings. Or, in a recorded seminar, the user may want to view a segment where participants discussed a certain topic. In a personal video collection, the user may simply seek a particular piece of footage. To meet these diverse needs, we describe three visual interfaces 1-3 that help people identify potentially useful or relevant video segments. In such interfaces, keyframes—still images automatically extracted from video footage—can distinguish videos, summarize them, and provide access points. Well-chosen keyframes help users select videos and enhance a listing's visual appeal. Our goal is to identify keyframes that describe an entire video and provide access to its relevant parts. Keyframe selection can vary depending on the appli-cation's requirements. For example, a visual summary of a video-captured meeting only needs a few keyframes that show the highlights, whereas a video editing system needs a keyframe for every clip. Browsing interfaces require a somewhat even distribution of keyframes over the length of the video. Selecting good keyframes is difficult. Most systems select at least one keyframe for each shot. A shot is captured video from a single camera running from camera on to camera off. Using one keyframe per shot means that representing a one-hour video usually requires hundreds of keyframes. In contrast, our approach for video indexing and summarization selects fewer keyframes that represent the entire video and index the interesting parts. The user can select the number of keyframes or the application can select the optimal number of keyframes based on display size, but a one-hour video typically will have between 10 and 40 keyframes. We use several techniques to present the automatically selected keyframes. A video directory listing shows one keyframe for each video and provides a slider that lets the user change the keyframes dynamically. The visual summary of a single video presents images in a compact, visually pleasing display. To deal with the large number of keyframes that represent clips in a video editing …